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Abstract 

We discuss the complex dynamics of a non-linear random networks 
model, as a function of the connectivity k between the elements of the 
network. We show that this class of networks exhibit an order-chaos 
phase transition for a critical connectivity fcc = 2. Also, we show that 
both, pairwise correlation and complexity measures are maximized in dy- 
namically critical networks. These results are in good agreement with 
the previously reported studies on random Boolean networks and random 
threshold networks, and show once again that critical networks provide 
an optimal coordination of diverse behavior. 



1 Introduction 

Random Boolean networks (RBNs) are a class of complex systems, that show a 
well-studied transition between ordered and disordered phases. The RBN model 
was initially introduced as an idealization of genetic regulatory networks. Since 
then, the RBN model has attracted much interest in a wide variety of fields, 
ranging from cell differentiation and evolution to social and physical spin sys- 
tems (for a review of the RBN model see [T] and [5], and the references within). 
The dynamics of RBNs can be classified as ordered, disordered, or critical, as 
a function of the average connectivity fc, between the elements of the network, 
and the bias p in the choice of Boolean functions. For equiprobable Boolean 
functions, p = 1/2, the critical connectivity is kc — 2. The RBNs operating 
in the ordered regime (k < kc) exhibit simple dynamics, and are intrinsically 
robust under structural and transient perturbations. In contrast, the RBNs in 
the disordered regime (fc > kc) are extremely sensitive to small perturbations, 
which rapidly propagate throughout the entire system. Recently, it has been 
shown that the pairwise mutual information exhibits a jump discontinuity at 
the critical value kc of the RBN model [3] . More recently, similar results have 
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been reported for a related class of discrete dynamical networks, called random 
threshold networks (RTNs) g]. 

In this paper we consider a non-linear random networks (NLRNs) model, 
which represents a departure from the discrete valued state representation, cor- 
responding to the RBN and RTN models, to a continuous valued state represen- 
tation. We discuss the complex dynamics of the NLRN model, as a function of 
the average connectivity (in-degree) k. We show that the NLRN model exhibits 
an order-chaos phase transition, for the same critical connectivity value kc = 2, 
as the RBN and RTN models. Also, we show that both, pairwise correlation and 
complexity measures are maximized in dynamically critical networks. These re- 
sults are in very good agreement with the previously reported studies on the 
RBN and RTN models, and show once again that critical networks provide an 
optimal coordination of diverse behavior. 



2 NLRN model 

The NLRN model consists of N randomly interconnected variables, with con- 
tinuously valued states —1 < x„ < -1-1, n — 1, ...,N. At time t the state of the 
network is described by an N dimensional vector 

x(t) = [xi(i),...,xjv(i)]^, (1) 

which is updated at time t + 1 using the following map: 

x(t + l) = /(w,x(t)), (2) 

where 

/ (w, x(i)) = [/i (w, x(0) , In (w, x(t))]^, (3) 

and 

fn (w,x(t)) = tanh WnmXmit) + Xp \ , n^l,...,N. (4) 

\m=l ) 

Here, w is an iV x iV interaction matrix, with the following randomly assigned 
elements: 



-1 with probability ^ 
Wnm = with probability ^^^^ , (5) 

2N 



-1 with probability ^ 



and k is the average in-degree of the network. 

The interaction weights can be interpreted as excitatory, if Wnm — 1, and 
respectively inhibitory, if Wnm = — 1- Also, we have Wnm — 0, if Xm is not an 
input to Xn- Obviously, the threshold xp can be considered as a constant input, 
with a fixed weight w„o = 1, to each variable x„. Therefore, in the following 
discussion we do not lose generality by assuming that the threshold parameter 
is always set to a;o = 0. 
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3 Phase transition 



In order to illustrate the complex dynamics of the NLRN system, we consider the 
results of the simulation of three networks, each containing N = 128 variables, 
and having different average in-degrees: k = 1, k = 2 and respectively fc = 4. 
Also, the continuous values of the variables Xn{t) are encoded in shades of gray, 
with black and white corresponding to the extreme values ±1. In Figure 1, 
one can easily see the three qualitatively different types of behavior: ordered 
{k — 1), critical (fc — 2), and respectively chaotic (fc — 4). 

A quantitative characterization of the transition from the ordered phase to 
the chaotic phase is given by the Lyapunov exponents [5 , which measure the 
rate of separation of infinitesimally close trajectories of a dynamical system. 
The linearized dynamics in tangent space is given by: 

(5x(< + l) = J(w,x(t))(5x(t), (6) 

where J is the Jacobian of the map /, with the elements 

dfn 



N 



1 - tanh^ ^ WnmXmit) 



(7) 



and 5ii.{t) is the separation vector. The dynamics of 5n.{t) is typically very 
complex, involving rotation and stretching. Therefore, the rate of separation 
can be different for different orientations of initial separation vector, such that 
one obtains a whole spectrum of Lyapunov exponents. In general, there are TV 
possible values, which can be ordered: Ai > A2 > ... > Xn- These Lyapunov 
exponents are associated with the Lyapunov vectors, U2, WAf, which form 
a basis in the tangent space. A perturbation along Vn will grow exponentially 
with a rate A„. Oseledec's theorem [5] proves that the following limit exists: 

A=limlln^. (8) 

t^oo t ||Mo)ll 

We should note that, Oseledec's limit will always correspond to Ai, because 
an initial random perturbation will always have a component along the most 
unstable direction, vi , and because the exponential growth rate the effect of the 
other exponents will be obliterated over time. Thus, in general, it is enough to 
consider only the maximal Lyapunov exponent (MLE) , which is enough to char- 
acterize the behavior of the dynamical system [5 . A negative MLE corresponds 
to an ordered system (fixed points and periodic dynamics), while a positive 
MLE is an indication that the system is chaotic. A zero MLE is associated 
with quasiperiodic dynamics and corresponds to the critical transition. Figure 
2 shows the MLE as a function of the average in-degree, A(fc). One can see that 
the critical in-degree is fcc = 2, such that for k < kc the NLRNs are ordered, 
and for k > kc the NLRs become chaotic. The numerical results were obtained 
by averaging over the NLRNs ensemble for each fc, using AI — 256 NLRNs with 
TV = 256 elements. Also, for each time series we have discarded the first 1024 
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steps, in order to eliminate the transient, and the MLE was calculated from the 
next 1024 steps. 

In order to provide a more detailed characterization of the order-chaos phase 
transition we introduce the following spectral complexity measure: 

- H^D^, (9) 

where is the spectral entropy, and D^j is the spectral disequilibrium. The 
complexity is defined by the interplay of two antagonistic behaviors: the increase 
of entropy as the system becomes more and more disordered and the decrease in 
the disequilibrium as the system approaches chaos (equiprobability) . A similar 
complexity measure, evaluated in the direct (time) space, was introduced in [7], 
for discrete state systems. In contrast, our complexity measure is defined for 
continuous state systems, and it is evaluated in the inverse (frequency) space. 

In order to define the spectral entropy [S], we consider the discrete Fourier 
transform (DFT): 

X„(c^) = F^[x„(<)] = [Xn{l),...,Xnmf, (10) 



T 

Xn{uj) ^^Xn{t)exp{-2niujt/T), uj^l,...,n, (11) 



t=i 

and the power spectrum: 

Y„(u;) = [y„(l),...,y„(f^)]^, (12) 

Yn{u) ^ X:iuj)Xniuj) ^ \XnH\\ io^l,...,n, (13) 
of the time series: 

x„(0 = [a;„(l),...,x„(T)]^, (14) 

corresponding to the attractor of the variable n of a given NLRN. Here, X* 
stands for the complex conjugate value. Since the variables Xn(t) are real, the 
DFT result has the following symmetry: 

X„(^) = X:{T - u), (15) 

and therefore the power spectrum Y„(a;) has only ~ T/2 positive values: 
One can normalize the power spectrum such that: 

M^)= ^o^"tl - = l,.-.,f^, (16) 

and 

n 

J2pn{0j) = l. (17) 

The new variable p„(w) can be interpreted as the probability of having the fre- 
quency uj embedded in the time series x„(t). Thus, using the spectral probability 
vector 

Pn{t0)^[pn{l),...,Pnmf, (18) 
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one can define the spectral entropy of the time series x„(t), as fohowing: 

^ n 

[Pn (w)] = - ^ p„ (w) l0g2 P„ (w) , (19) 

where log2 is the normahzation constant, such that < < 1. 

Obviously, the spectral entropy of the ordered systems will be low, ~ 0, 
since only a very small number of frequencies are present, while the spectral 
entropy of chaotic systems will be high, Hi^ ^ 1, since a large number of fre- 
quencies are present. The spectral entropy takes the maximum value, = 1, 
for the equilibrium state, which is defined deep in the chaotic regime, where all 
frequencies become equiprobable: p{uj) — co ~ 1, Q. 

The spectral disequilibrium of the time series x„(t), measures the displace- 
ment of the corresponding probability vector p„(cj) from the equilibrium state, 
and it is defined as following: 

n 

D^[Pr,{u),n-^]^Y.[p^{w)-n-^]^. (20) 

A special attention is necessary in the case when the attractor is zero: x„(t) — 0. 
In this particular case, the power spectrum is also zero, Y„(a;) — 0, and the 
probability vector p„(aj) is undetermined. In order to overcome this difficulty, 
we define = and = 1 for this particular attractor, such that it has the 
lowest entropy and the largest displacement from equilibrium. 

Since the spectral disequilibrium measures the distance between two distri- 
butions, one may consider also the spectral Kullback-Leibler divergence [9] as 
an alternative. However, for the considered NLRN model, the Kullback-Leibler 
divergence is simply given by: 

i?5^[p„(c.)||17-i] = ^f]p„(a.)log2(^^) =l-i?.[p„(w)]- (21) 

Similarly, one can show that the symmetrical Kullback-Leibler divergence is 
given by: 

n 

[p„ (c^) 1 1 n-i] + D^^ [O- 1 1 |p„ {lu)] = ~H^ [p„ (c.)] - — — ^ log2 pniuj). 

(22) 

Therefore, in this case, the Kullback-Leibler divergence (or its symmetrical ver- 
sion) can be expressed in terms of entropy. Thus, the spectral disequilibrium 
seems to be a more appropriate distance measure, since it cannot be expressed 
in terms of entropy. 

Another quantity of interest is the pairwise spectral correlation between the 
power spectrum of two network variables n and m, which is defined as: 

rv V 1 — ~ ^■n-)'^O^m ~ ^m) ^„„n 
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where Y„ and represents the mean values. 
The average correlation for a given NLRN is: 

N N 

where we have excluded the self-correlation terms {S{n,m) — 1 ii m — n and 
6{n, m) = if m 7^ n). 

In Figure 3 we give the numerical results for the above spectral measures 
(entropy, disequilibrium, complexity and correlation), obtained by averaging 
over the NLRNs ensemble (M = 256 networks with N — 256 elements and 
T = 1024). One can see that both the complexity and the correlation measures 
are maximized by the critical NLRNs with fee = 2. 

As mentioned at the beginning of the paper, the continuous NLRN model is 
directly related to the binary RTN model, which has been extensively studied 
recently [3], [TU]. Recently, we have investigated the binary RTN model, using 
similar quantities, complexity, entropy, and the mutual information, which are 
well defined in the time domain. The obtained results for both NLRN and RTN 
models are in very good agreement, showing a phase transition for the same 
critical connectivity kc = 2. Also, for the RTN model, we have shown that the 
mutual information, which is the binary counter part of the spectral correlation, 
is maximized for fcc = 2. Similar results have also been previously reported for 
the RBN model [3]. 

4 Conclusion 

We have shown numerically that the NLRN model exhibits an order-chaos phase 
transition, for the same critical connectivity value kc — 2, as the RBN and 
RTN models. Also, we have shown that both the pairwise correlation and the 
complexity measures are maximized in dynamically critical networks. These 
results are in very good agreement with the previously reported studies on the 
RBN and RTN models, and show once again that critical networks provide an 
optimal coordination of diverse behavior. We would like also to note that these 
optimal properties of critical networks are likely to play a major role in biological 
systems, perhaps serving as important selective traits. Given the potential 
biological implications, it is of interest that recent data suggest that genetic 
regulatory networks in eukaryotic cells are dynamically critical 11,. Also, recent 
experiments conducted on rat brain slices show that these neural tissues are 
critical |12| . Thus, it seems plausible that in cells, neural systems, and other 
tissues, natural selection will have acted to maximize both the correlation across 
the network, and the diversity of complex behaviors that can be coordinated 
within a causal network. Ordered networks have convergent trajectories, and 
hence forget their past. Chaotic networks show sensitivity to initial conditions, 
and thus they too forget their past, and are unable to act reliably. On the 
other hand, critical networks, with trajectories that on average neither diverge 
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or converge (quasiperiodic dynamics), seem best able to bind past to future, 
and therefore to maximize the correlated complex behavior. 
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Figure 1: Three qualitatively different types of behavior or the NLRN model: 
ordered {k = 1), critical (A: = 2), and respectively chaotic (fc = 4). 



8 



0.5 



0^ 



-0.5 







-1.5 



Figure 2: The maximal Lyapunov exponent of the NLRN model, as a function 
of the connectivity: A(fc). 




Figure 3: The spectral measures of the NLRN model as function of of connec- 
tivity: entropy Huj{k), disequilibrium Duj{k), complexity Qw(fc), and correlation 
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